A multi-encoder variational autoencoder controls multiple transformational features in single-cell image analysis

Image-based cell phenotyping relies on quantitative measurements as encoded representations of cells; however, defining suitable representations that capture complex imaging features is challenged by the lack of robust methods to segment cells, identify subcellular compartments, and extract relevant features. Variational autoencoder (VAE) approaches produce encouraging results by mapping an image to a representative descriptor, and outperform classical hand-crafted features for morphology, intensity, and texture at differentiating data. Although VAEs show promising results for capturing morphological and organizational features in tissue, single cell image analyses based on VAEs often fail to identify biologically informative features due to uninformative technical variation. Here we propose a multi-encoder VAE (ME-VAE) in single cell image analysis using transformed images as a self-supervised signal to extract transform-invariant biologically meaningful features, including emergent features not obvious from prior knowledge. We show that the proposed architecture improves analysis by making distinct cell populations more separable compared to traditional and recent extensions of VAE architectures and intensity measurements by enhancing phenotypic differences between cells and by improving correlations to other analytic modalities. Better feature extraction and image analysis methods enabled by the ME-VAE will advance our understanding of complex cell biology and enable discoveries previously hidden behind image complexity ultimately improving medical outcomes and drug discovery.


Supplementary Figure 3: Extracted biological metrics from CYCIF.
Cell intensity maps were circularized to allow easy compartmentalization. The inner, middle and outer mean intensities were extracted by dividing the cell into thirds radially. The mean intensity of the whole cell was also taken. The radial mean intensity map was created by taking the average intensity for each radius across the circularized cell. The slope of the radial mean intensity map was then taken to create a single metric for stain distribution.
Supplementary Figure 4: Regional cell images across UMAP visualization. a) UMAP embeddings for respective VAE encodings, allowing for qualitative visual evaluation of ligand separability. b) Regional cell images were sampled from locations throughout UMAP space to highlight the differences in expression pattern. Stains shown were selected based on a combination of being correlated to important VAE features and hand-selection for known variance. Scale bars represent 40µm.

Supplementary Figure 5: Standard VAE feature aggregation and transitive intermodality correlation. a)
Using the single cell observations as features, correlations are drawn between pairs of standard VAE features. These features are then hierarchically clustered to observe patterns and reduce VAE features to aggregated feature sets. Cell images were assigned aggregated feature scores using the mean expression of each feature in a cluster. Shown are representative cells that are highly expressing for each respective cluster. Scale bar represents 20µm for all single cell images. b) Correlation matrix between RPPA pathway activity scores and standard VAE aggregated features. Samples from the two modalities were paired by their ligand treatments, resulting in a sample size of n=6 biologically independent ligand treated cell populations. RPPA pathways and VAE features were hierarchically clustered to show prominent patterns in correlation. Standard VAE aggregated features were also correlated to several metrics of CYCIF expression (mean inner, mean middle, whole cell means, and radial slope) for all 23 stains. This CYCIF correlation was done using the full dataset of single cell images (sample size n=73,134 single cell images)). The table of CYCIF correlations shows the top three correlations for each ME-VAE aggregated feature. Aggregated feature 4 shows high correlations to almost all RPPA pathways (3 rd column from the right), and the DNA death/repair and apoptosis pathways also has high correlations to almost all aggregated features (1 st and 5 th rows).

Supplementary Figure 6: Representative cell images for each ligand treatment.
Representative cell images are shown for each ligand treatment (rows) and are shown using several stains (columns). Each column also includes a # that ties back to the multi-encoder feature that is highly correlated. Scale bar represents 40µm for all cell grids.

Supplementary Figure 7: Separability of ligands using aggregated ME-VAE features.
Density function for several CYCIF and ME-VAE feature pairs. A two sided ANOVA was performed for a features and intensities between populations in order to compute the F statistic and p-value ( PR(>F) ). Subsequently, the mean Tukey-pairwise p-value across ligands and mean effect size shown for each feature. ME-VAE features used for comparison were the features with largest correlation to the respective CYCIF marker. This analysis utilized all 73,134 cell images from the MCF10A dataset.